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The pipelines of approaches for classifying diabetic retinopathy were 
examined in this study. The effort entails developing appropriate 
transformations and estimators that can be used to automate the process of 
diabetic retinopathy detection. The segmentation of the blood vessels was 
done using a hybrid algorithm that uses Otsu and median filter to get the 
region of interest. Further, ten classifiers were investigated in order to 
develop an automated pipeline for diabetic retinopathy detection. The ten 
classifiers were reviewed based on earlier work in a similar setting and on an 
exploration of new ways for identifying diabetic retinopathy. To overcome 
the challenge of low volume of dataset, data argumentation was done so that 
a generic classifier can be configured. Extensive hyper parameter tuning was 
performed, and it was shown that the gradient boosting approach is the most 
stable technique for detecting diabetic retinopathy. This was validated using 
a 10K fold cross validation method on many metrics (accuracy, recall, 
precision, and v-measure score). Hyper-parameter tuning helped in 


achieving accuracy of 0.96. 
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1. INTRODUCTION 

The world is suffering from health problems due to covid-19 pandemic [1]. India is no exception. 
Indians are in-fact, quite prone to COVID-19 virus due to multiple medical reasons [2]. One of the many 
reasons is that a large number of people in India are diabetic. India can be considered as a “Diabetic Capital” 
of the world [2]-[4]. It is a clinically well-established fact that diabetes creates many severe medical 
conditions as a comorbid [4], [5]. The co-prevalence of comorbidities increases with increase in biological 
age as well. These facts are clinically known as a sequel of diabetes [6]. COVID-19 pandemic is posing a lot 
of challenges to health systems and it urgently needs to be supplemented with emerging proven technologies. 
And these technologies must work for both healthcare providers and patients. One of the most critical and 
challenging situations is to manage the co-morbidities of covid-19 patients, including diabetes and associated 
ailments such as diabetic retinopathy (DR). Due to this, diabetic retinopathy is spreading in India at a 
considerable speed [7], [8]. For the medical diagnosis and treatment of the diabetic retinopathy, growth of 
imaging technology is necessary, and at the same time leveraging the latest machine learning algorithm is 
essential. The landscape of the technologies and algorithms that support detection of diabetic retinopathy 
begins with the advent of machine learning algorithms [9]-[11]. These days for detection of diabetic 
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retinopathy, use of fundus photography in diagnoses has become quite reliable. The current literature shows 
that the most frequently employed algorithms in context of detecting diabetic retinopathy include neural 
networks, convolutional neural network (CNN) [12]-[14], support vector machine (SVM) [15], [16], 
K-nearest neighbors (KNN) [15], decision trees [17] and regression based algorithms. In almost all cases, the 
input dataset is either pre-processed or transformed into a standardized/normalized form. Besides, some 
authors have used combinational or hybrid approaches for detection of diabetic retinopathy. There is an 
imperative need to develop frameworks and pipelines of algorithms that can restore the reliability of such 
existing algorithms. This is because the detection of the diabetic retinopathy involves examination of 
multiple factors such as exudates [18], [19], optic nerve health [20], blood vessel shape, and observation 
related to size of other eyeball parts [21], [22]. The imaging helps to clinically corroborate the symptoms of 
diabetic retinopathy and at the same time helps to ascertain the health of the blood vessels in terms of their 
shape, formation, and thickness [23], [24]. More importantly, the imaging technology has been in use for a 
long time in monitoring the brain and eye in terms of various medical conditions and disorders. However, 
diabetic retinopathy is a condition in which it is of utmost importance to have a tool that can ascertain the 
condition of the brain along with the eyes. This is because of diabetic retinopathy is not a disease that can be 
diagnosed without doing the visual acuity test on the eye as well as an ophthalmologist examination. Hence, 
an imaging tool that can ascertain the condition of the eye and brain at the same time is the most sought-after 
tool for monitoring the condition of the brain and eye. The imaging technology helps to find if some exudates 
are blurring the vision of the person suffering from diabetic retinopathy [25], [26]. It helps to spot any 
abnormalities in the eye with a lot of ease and accuracy. However, at the same time, it should be noted that 
from digital images only relevant information is required to be extracted and segmented, so that mapping 
with real life medical conditions could be done. Digital fandus photography enables us to do the diagnosis in 
an effective way [27]. The standard fundus imaging process includes capturing eye image in posterior pole 
position at 30 degrees [28], [29]. It can help to examine the optic nerve and macula properly. However, in 
some cases, it has been observed that this imaging modality cannot verify clinically noticeable differences. 
Hence, some additional pre-processing of the images may be required to build a clinically relevant solution 
for the detection of the diabetic retinopathy [30]-[32]. Images may suffer from the effects of angle distortion 
and artificial artifacts and development of detection will require pre-processing of images. Another method, 
stereoscopic fundus photography is employed when there is a need to medically examine both eyes and it has 
been observed in many cases that for tracking the treatment outcomes it is important to examine both the eyes 
at different stages. The challenge is if we use eye images to have noise, distortion, for building a diabetic 
detection system, a lot of computation complexity needs to be resolved for building a DR detection system. 
This problem may be solved by using state of art machine-learning algorithms. It can also be observed that 
clinical dataset in the context of DR in the public domain suffers from issues such as small in size, unmarked 
and unannotated, imbalance and non-specific in terms of demographics [33]-[35]. Due to these issues the use 
of deep learning algorithms that extract image features directly is also in use. At the same time, it can be 
observed that reliability of the machine learning algorithm is proven beyond doubt in this context of detection 
DR. In fact, it can be safely stated that with the use of handcrafted features methods, conventional statistics, 
data fusion, data argumentation, transfer, active learning models the landscape is not complete. There are 
ample research studies in this context; however, the problems related to imbalance and small size of the 
dataset remains alive as we undertake this study [36], [37]. 

The objective of this research work is to construct a numerically stable classifier that can classify 
DR. This manuscript provides mechanisms for detecting DR using image based features and machine 
learning algorithms. To accomplish these goals, an explorative journey for building a classifier that can 
detect DR using eye fundus images was performed. The exploration will be focused on finding an 
appropriate array of methods that can be classified using a limited number of instances of eye images. The 
machine learning models that are going to be put under evaluation will be based on the foregoing work done 
by many authors in the context of building diabetic retinopathy classifiers and the challenges such as small 
size of dataset. At the end of work, an attempt to overcome the inadequacies of existing algorithms in terms 
of accuracy is demonstrated. The inadequacies will be overcome by conducting extensive hyper-parameter 
tuning and research oriented exploration. 


2. PROPOSED METHOD 

This section gives detailed information on all the steps taken to construct the most stable and 
accurate machine model for detecting DR. The flow of the machine learning algorithms/classifier can be 
streamlined with the support of a concept known as “Pipelining”. The process of pipelining in machine 
learning is typically used for automating the standard functions that are required for construction of an 
accurate classifier system. In the context of our research work, the objective is to build an automated 
assortment of functions that can help classify a healthy eye from an unhealthy eye i.e. an eye suffering from 
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diabetic retinopathy. The original database contains a total of 30 high-resolution fundus images (15 healthy 
and 15 images of patients suffering from diabetic retinopathy). The reference to the dataset can be found here 
[38]. The quantity of the dataset is a significant hindrance to the creation of machine learning models. The 
largest obstacle in this research has been the dataset's short size. As a result, the generalization of the 
machine learning model becomes more challenging. Simultaneously, it may result in under-fitting. To 
address this obstacle, data augmentation and 10K cross-validations were used to verify the construction of a 
highly credible model. Ten distinct augmentation functions (flip, rotation, brightness, and zoom.) were used 
on the dataset. This way we extended the dataset size to 300 images ((healthy and unhealthy)) for both 
classes. The second technique for developing a reliable classifier is to keep the number of attributes to a 
maximum. Figure 1 shows sample images used in this research work after the segmentation process. 


Figure 1. Segmented sample images 


2.1. Segmentation of the blood vessels 

An exploratory study was conducted in this research work to determine which segmentation 
method(s) can accurately segment the blood vessels of images of both (left and right) eyes. The segmentation 
was performed in order to obtain the regions of the eyes that correspond to the blood vessels. This current 
work employs the same methodologies as in [39], [40]. Explicitly, Otsu global thresholding [41], [42] with 
the mean filter for extraction of the blood vessels worked best. For extraction of the features and attributes 
for the machine learning algorithms region props method was used. The region props method treats the 
negative and zero values pixels as background and other pixels are marked as one. It checks that at least four 
pixels are connected with each other and it can be considered as a region. Each region is given a numerical 
label or index and geometric and statistical properties of each region can be extracted. The geometric 
property such as area, convex area, and can give hints about the state of the eye [43]. The properties include 
area, filled area, and convex area, centroid, bounding box area, orientation, major axis length, minor axis 
length, eccentricity, moments normalized, inertia tensor, inertia tensor eigen-values, and label. The selection 
of these attributes/properties has been done on the premise that when an eye is examined in detail (region 
wise), changes in the shape, and size can be observed. These changes (morphology) can be observed by 
measuring the area, volume, eigen vectors, and other statistical properties of the region of interest. In the 
coming sections, discussion on the next step i.e. model selection in building a pipeline. For doing model 
selection ten classifiers (logistic regression (LR), SVM, KNN, linear discriminant analysis (LDA), 
classification and regression trees (CART), extra trees classifier (ETC), Naive Bayes (NB), random forest 
(RF), gradient boosting classifier (GBC), and multilayer perceptron (MLP)) [44]-[50] of different types have 
been selected so that a robust pipeline can be constructed for detection of DR. Secondly; this covers some of 
the work done by previous authors as well. However, in this part of the paper, a discussion on the impact of 
scaling on accuracy of classifiers followed by the interpretation is done. For best possible performance, some 
algorithms require normalization so that all the attribute values have a similar range while some algorithms 
require data that should be standardized. In this subsection, we evaluate the impact of the different scaling 
methods on the accuracy of the classifiers using a 10K fold cross validation method. 


2.1.1. Interpretation 

From the outcome of all these algorithms and after the application of different scaling methods, it is 
clear that the data contain a large number of extreme values and each attribute has a different scale (Refer 
Figure 2). This can be attributed to the fact that the accuracy is getting impacted due to change in scales of 
the data. These two issues lead to degradation of predictive performance [46]. Gradient-based classifiers 
assume that the data is approximately standardized and other classifiers have some other underlying 
presumption of the distribution of the data. Quantile transformers [47] and power transformers perform non- 
linear transformation of the data. Quantile scaling is based on the minimization process between the outliers 
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and inliers data points. The power transformers map its data to normal distribution so that variance and 
skewness is minimized. It can be observed from Figure 2 that non-linear transformation [48] (power) is 
helping to achieve high level accuracy in case of GBC (0.91). The standard deviation is also tight (0.1500). It 
seems that MinMax and MaxAbs scaling [49], [50] is not working in favor of accuracy. This may be 
attributed to the fact that in MinMax scaling the data is rescaled in range of (0, 1) and compresses all the 
inliers into a narrow range of [0, 0.005]. The robust scaling however is performing even better than the power 
transformer by 2% for the GBC algorithm. However, for other classifiers the accuracy is less as compared to 
the power transformer scaling. Normalization [51] does not have a good impact on the balanced accuracy of 
the classifiers. Therefore, it is imperative to use robust or power transformation scaling for doing data 
preprocessing and preparation of feature data for building a reliable classifier. The evaluation of the impact 
of training data size shows interesting facts. Since, the number of eye images is just 30; it is difficult to 
determine the level of generalization that can be achieved by these classifiers. Hence, to make a generalized 
algorithm and numerically stable we are relying on a large number of attributes as input data for machine 
learning algorithms. Consequently, we continued with the 139 input features (i.e. geometric properties of first 
fifteen objects) to find what percentage of training data is enough for building a classifier pipeline. 
Furthermore, after the data argumentation processes the dataset size increased to 300 samples keeping 
generalization as an important goal in mind. 

It can be observed from Figure 3(a) and Figure 3(b) (in Appendix) that when the testing ratio is 15% 
the classifier’s accuracy remains stable. This is an indication 85% of data is required for maintaining a good 
level of balanced accuracy of almost all the algorithms. In this next section, we evaluate the other 
performance parameters such as recall, precision, and so that final selection of the machine-learning model 
can be made and construction of the automated pipeline can be made. 


Algorithm Performance Based On Scaling Methods 
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Figure 2. Impact of scaling on accuracy 


3. METHOD 

In this section, we conduct an evaluation of important metrics for final selection of the machine- 
learning model. Figure 4(a) and Figure 4(b) shows the performance analysis of the classifiers. The 
interpretation is followed thereafter. 

The metric R2 [52] helps to find the goodness of the machine learning model. This metric helps to 
find out how good the model is in terms of fitting (fit, over or under) the data. It can be observed that the 
GBC and CART algorithm has positive R2 value (0.5) with tight standard deviations. The value in case LDA, 
KNN, and ETC is negative showing that the machine learning model is not choosing the correct trend for 
identifying class boundaries. Since, this work is medical related, the best scenario will be “high recall and 
high precision”, because the cost of not diagnosing a person with DR may lead to worsening of the medical 
condition of the person if he is actually suffering from DR. At the same time, diagnosing a person with DR, 
who is not suffering from DR may lead to giving him or her false treatment and unnecessary medical 
complications. Hence, in the context of our research work it is important to have a good trade-off between 
recall and precision. Moreover, it is always desired that algorithms should be able accurately classify DR and 
non-DR cases for the full dataset and with small samples. This can be observed in the case of GBC and 
CART algorithms. The standard deviation of GBC is smaller than the CART algorithm, showing that GBC is 
a bit better in terms of stability. Furthermore, it can be observed in case of precision [53], that in the case of 
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GBC that it is able to acquire the highest level of precision. From this, it can be inferred that true positives 
are highest in the case of GBC and it has the lowest false alarm rate. It is evident that GBC will have the 
highest accuracy level followed by CART, as it will have similar values as those of recall and precision in 
context of V-measure metric. It is always desired that a class should have homogenous members and at the 
same time be different from the other class members while being assigned labels by the clustering or 
classification algorithm-measure is a metric that gives the harmonic mean of the homogeneity (all data 
members belong to respective class) and completeness (all data points are covered and assigned a class). It 
can be observed that the GBC algorithm has value 0.8, which is closest to the ideal value of one, followed by 
CART. Again, here also, it can be observed that GBC is offering better results as it has a tighter level of 
standard deviation. It can also be observed that SVM has the lowest level of performance. MLP, RF, and NB 
are slightly better than SVM. From this, it can be inferred that not all types of regression, probability and tree 
based classifiers are suitable for this task. There is a need for either to construct a new algorithm or improvise 
the existing algorithms such as GBC for better level of accuracy and precision. But before making a decision 
for making a new algorithm, a deeper level of investigation can be done on the best performance so that a 
valid reason for making a new algorithm can be taken in due course of this research work. In the next 
subsection, finalization of the classifier pipeline is done. This involves a hyper-parameter tuning of the GBC 
algorithm along with validation using 10 K-fold. Technically it includes: selection of the appropriate 
parameters, their ranges and running a grid search for finding the best possible combination of parameter 
values. The grid search process involves running a GBC algorithm with various permutations and 
combinations of parameter values to find a point at which the accuracy, recall, and other performance metrics 
do not further yield to higher levels of performance. This way a new machine learning model is constructed 
which is optimized, numerically stable and consistent with its performance as it is also validated using a 10K 
fold method. 


Algorithm Performance Analysis 
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Figure 4. Impact of R2, recall, precision and V-measure (a) performance analysis of classifiers and 
(b) performance analysis of classifiers (with standard deviation) 
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3.1. Finalization of pipeline 

It is apparent from the above hyper parameter tuning process using grid search method [54] and 
analysis that GBC is the best performer in terms of recall, precision, accuracy, and v-measure scores. At each 
step, we are using a 10K fold cross validation method to arrive at a decision. The 10 K-fold validation provides 
average values of the metrics using the inter quartile range method. The advantage is unbiased average 
calculation based on ten rounds of evaluation. From the scaling impact analysis, it has become apparent that 
normalization and standard scaling does not help, in-fact robust scaling or non-linear transformation (power) 
helped to improve the accuracy level of the classifiers. It was also found that train-test data split ratio also 
influences the accuracy of the classifiers. Again, through the process of 10 K-fold cross validation process it 
was found that a minimum of 0.85% available data is required for building a reliable classifier. Finally, the 
recall, precision, accuracy and v-measure score were computed using a 10 K-fold cross validation process to 
check if the GBC algorithm is best among these algorithms. In this section, prediction quality of the model is 
checked using different hyper-parameters. There are three kinds of hyper parameter variables that impact the 
working of gradient boosting performance. The first category is tree specific parameters that affect the 
performance of each individual tree and the second one impact the performance of operations of boating trees. 
Last but not the least, is miscellaneous parameters that support the overall performance of the GB algorithm. For 
fine tuning the algorithm specific range or value are given input to the algorithm and grid search method is 
involved based on specific goals. In the context of this optimization of GB algorithms the goal is to find 
optimized values of parameters which would yield the highest possible level of accuracy in classification. The 
Table 1 (in Appendix) gives details on all the hyper parameter options available and the optimized value found 
by the grid search method of fine tuning algorithms. 

It can be observed from Table 1 that the lower learning rate (0.15) with minimum number of 
features (sqrt) is the best possible combination to get the maximum accuracy. The hyper-tuning process 
showed that the exponential loss function provides better moderation and accuracy than the ‘deviance’ 
function. It can also be observed that the tree does not have to be too deep for getting maximum level of 
accuracy, as the optimized value is three. Since, GBC is a combination algorithm that takes feedback from 
multiple classifier models, the determination of the number of models required to get maximum accuracy is 
also important. Higher number of estimators means high level of overload and higher level of prediction 
latency. Nevertheless, it can be observed that it took 10 estimators to get a 0.96 level of accuracy. Adding 
more estimators did not yield higher order of accuracy. It implies that GBC may require some kind of 
modification or improvisation further to improve performance from this benchmark. 


4. RESULTS AND DISCUSSION 

The purpose of this research work was to build a pipeline of methods that directly help to detect DR. 
We started with the exploration and evaluation of 10 algorithms for this purpose. All these algorithms were 
given training with the fundus image’s image geometric features. Each eye image was analyzed at small 
region levels. Using the region prop method, the geometric properties of each region were extracted and 
compiled to form a feature dataset from the machine learning modeling. The preliminary evaluation of the 
entire algorithm set showed that gradient boosting algorithm is the best performer in terms of recall, 
precision, and accuracy. Hence, further exploration was done so as to ascertain the numerical stability and 
reproducibility of the GBC algorithm by using a 10K fold cross validation method. It can be observed from 
Figure 5 that the accuracy of the algorithm is improved by approximately 6%. 
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Figure 5. Accuracy levels with respect to pipeline (base model=0.901, with normalization=0.68, scaling 
(robust|power)=0.93, optimized model=0.96) 


Hyper parameter tuning based gradient boosting algorithm for detection of diabetic ... (Parul Datta) 


820 o ISSN: 2302-9285 


This research demonstrates that the best method for building an accurate DR classifier depends upon 
the type of scaling done on the dataset and the level of optimization we can do on the gradient boosting 
algorithm. Normalization does not help; in fact, it leads to deterioration of the performance of almost all the 
classifier algorithms. In the case of the gradient boosting classifier, the accuracy fell from 0.90 to 0.68 due to 
normalization. Nevertheless, the use of robust or power scaling improved the accuracy of the gradient boosting 
classifier, up to 0.93. The size of the dataset was small, due to which our strategy was to add more features or 
keep the same number of features so that maximum information can be provided to the machine learning 
models for getting trained. Hence, 139 features were used for evaluation of all the ten algorithms. Keeping this 
in mind, the task of optimization of GBC was done. The gradient boosting algorithm gives nine parameters for 
optimizing the performance of the base model. Full grid search method was used for finding most optimized 
hyper-parameters. The diagram in Figure 6 summarizes the pipeline of building DR classifiers. 


Geometric Region Gradient Optimized 
— wise Feature Boosting DR 
Extraction Algorithm Classifier 


Figure 6. Final pipeline for building DR classifier 


Raw Image 
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5. CONCLUSION 

An exploratory experimentation regime was followed and it was found the GBC algorithm is the 
most appropriate machine-learning model for building a classifier that can detect DR. For selection, the best 
performing and numerically stable model, a deeper investigation on the internal working of GBC models was 
done and it was found that some level of hyper-parameter tuning is required. The hyper-parameter tuning 
helped to achieve an accuracy level of 0.96. This work demonstrates that by following the appropriate 
pipeline of methods a highly accurate algorithm can be constructed. This work also showed that image 
geometric features can be effectively used for building a DR classifier, especially, when the dataset is small. 
For future directions, it is suggested that by understanding the nature of the decision rules, the GB algorithm 
can further be customized to suit the diabetic retinopathy classification. It is suggested that the pipeline of 
classifiers built in this context, be renewed with larger volume and size of labeled dataset. 
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Figure 3. Impact of training data size on accuracy (a) accuracy analysis based training ratio 
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Figure 3. Impact of training data size on accuracy (b) accuracy analysis based training ratio (standart 
deviations) (continue) 


Table 1. Gradient-boosting parameters 


Parameter How it helps to tune Input ne Interpretation 
1 Criteria Check the quality of the split. Mean Friedman friedman This function is most 
squared error, mean absolute error and Mae, _mse appropriate for finding 
frieman are supported values by GBN. Mae optimal solutions 
2 Learning rate Determines the impact of each tree on the {start, 0.15 The algorithm found that 
overall performance of GBM. Lower value end=0.15 maximum of 10 estimators at 
means higher degree of robustness and -2} a level of 0.15 learning rate is 
vice versa, secondly it determines the best value for getting 
trade-off between learning rate and no of maximum accuracy 
estimators. 
3 Loss function The loss function to be optimized (price to {multino exponen Minimization of the 
be paid for get maximum possible mial tial exponential loss function 
accuracy), there are two possible options in deviance yields a optimized solution 
GNB, deviance (optimize using logistic loss 
regression with probabilistic output) and function, 
exponential (a convex method that grows exponenti 
in such manner it makes algorithm more al loss 
sensitive to extreme values) function } 

4 Max depth Maximum depth of the tree. Deeper depth {values= 3 This implies that 3 is the 
means that higher levels of overhead 3,5,8} maximum limit for creating 

nodes. 5 or 8 are not viable 

5 Max features It is the minimum number of features that As thumb sqrt This implies that 

should be considered while making splits. tule 

Lower number implies less overhead. square 
root of 
the total 
features 
works 
well 

6 Minimum Sample Minimum number of observations required {0.1,0.2, 0.20 This implies 
leaf: in terminal node/leaf. If the data is not 0.3} for leaf nodes to be created a 
ceil(min_samples_ imbalanced, higher values are expected. minimum of 20% samples are 
leaf * n_samples) required. 

T Minimum sample It is the minimum number of observations {start, 0.13 The value is rounded off with 
split that are required by node for further data end= the ceil function and then a 
ceil(min_samples_ splitting. It helps to control over/under- 0.10, number of observations are 
split * n_samples) fitting. 0.20} computed. 

8 Number of Number of sequential trees. Higher values {start, 10 Value of 10 means that a GB 
Estimators means large overhead. It works on the end= classifier requires at least 10 

principle of ensemble i.e combines a setof 5, 10} estimators for getting 
weak learners for delivering prediction maximum possible level of 
accuracy but at the same time we do not accuracy. 

want too much overhead. 5 is the minimum 

number of estimators and 15 is the 

maximum number. 

9 Subsample Fraction of observations to be selected for 0.9,0.5,0. 0.9cc This means at least 90% 

each tree. Optimal values are required so 33 samples are required for 


that accuracy is not compromised. 


building a tree. 
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